A Reduced-Reference Method for Characterizing Color Noise in Natural Images Captured by Digital Cameras

Noise is one fundamental quality attribute in digital cameras. Traditionally, noise has been measured from solid patches of artificial test targets. In image quality research, it has been difficult to find connections between a test target and subjective test data. In addition, image quality algorithms computed from natural images are not well correlated. In this paper, we propose a novel approach for measuring color noise from natural images. With the proposed method, the suitable surfaces for noise calculations are located from the scene using a reference camera image. It is now possible to use the same image files for subjective and objective measurements and correlations are easier to find. The results show that the method is promising. Its performance was better for predicting subjective noise compared to the visual noise metric that is the state-of-the-art test target method for digital cameras. Introduction Digital cameras produce different noise types in images. For example, noise can be high-frequency achromatic noise, lowfrequency red-green or yellow-blue color noise or a combination of both. In this study, a new method to measure and characterize color noise directly from a natural image is described. The proposed method is based on a reference camera. The reference camera shoots a natural scene, and the appropriate areas are identified from the image for measurement purposes. The method has been developed for camera benchmarking studies. The method requires that the images of the reference camera and the cameras to be benchmarked are produced from the same scene. The study of color noise in the literature can be divided in two distinct areas. In the first area, the goal is to describe the noise model and weighting factors of its chrominance components. In these studies, noise level has often been measured from solid patches of specific test targets. For example, Kuang et al. [2] fit the parameters of the noise model based on empirical data. They also implemented a function incorporated in the noise model that described the effect of luminance level. In another study, Kelly and Keelan [3] described new weighting factors of the chrominance component for the signal-to-noise ratio calculation. In the second area of color noise study, the goal is to find the noise level or noisy areas from natural images for noise reduction purposes. Gheorghe et al. [5] proposed a method to reduce color noise from a natural image. Their method was based on a hybrid multi-scale spatial dual tree adaptive wavelet filter in huesaturation-value color space. Lee [4] proposed a method to detect color noise areas from natural images. His method was based on correlation between the R/G/B color channels. In addition, a noise metric for luminance channel has been proposed [10]. These methods are based on the no-reference (NR) approach. The measurements are performed without the original noiseless images. The problem with using NR methods with digital cameras is that these methods are often sensitive to other image distortions. For example, NR noise metrics can interpret image details as noise energy. In addition, NR metrics are often highly image-content specific. The proposed method differs from the earlier methods discussed in the literature. The method is based on the reducedreference (RR) approach. It utilizes information from a reference image, but it does not need a pixel-wise equivalence as the fullreference (FR) approach does. Pixel-wise comparison is not even possible when digital cameras are benchmarked. When images are produced from a given scene using different digital cameras, there is always rotation, scaling and 3D-projection between the images. We can find an analogy between the test target method and the proposed method. With the test target method the properties of the solid patches are known. With the proposed method, the suitable areas are located for measurements from the scene using a reference camera image. The selection is based on distortion. In this study, we describe how surfaces for noise calculations can be selected. In addition, we show how noise type can be characterized and noise level can be measured from these surfaces. The benefit of the proposed method compared to test target methods is that the same images can be used for subjective and objective measurements. It has been difficult to find correlations between test target computations and subjective test data such as MOS using conventional image quality research methods. We believe that these relationships are easier to find if both measurements are made using the same natural images. Compared to NR methods, the benefit of the proposed method is that at least some features from the reference (noiseless) image and scene are known. With these features, the problems related to the other image distortions and image content can be avoided. Method The proposed method is based on blocks that are located from the scene using a reference camera image. The block selection is based on three features: chromatic energy, achromatic energy and brightness of the block. The chromatic energy of the blocks should be low. The blocks can have achromatic structural energy, but this structure should be composed more of random texture than edges. There are two reasons why random texture in a scene can be beneficial for noise measurements. The first reason is that achromatic texture-like surfaces in scenes are sensitive to color noise in digital camera images. The second and more important reason is that texture-like surfaces present challenges for noise reduction algorithms in cameras. If the structure is edge-like, then a noise reduction method can easily filter the noise away from the neighboring smooth area of the edges. If the structure is a random texture, then it is difficult to separate the noise energy from the image structure energy using computational methods. In addition, the intensity of the selected blocks should not be too low or too high. If a block is too bright, then it becomes saturated for images produced by low-end cameras. If the block is too dark, then it is possible that a low-end camera does not detect its structure energy and that the camera image processing software applies strong noise reduction to it. The method was applied in the YCbCr space. With opponent color space, it is possible to separate achromatic information from chromatic information. The method operates on the principle that the control blocks are initially symmetrically located on reference image (Figure 1a). The method searches for new locations for the blocks on the limited neighborhood in Cb and Cr channels by maximizing the homogeneous metric value. Figure 1b shows the blocks that are located on the new places for the Cb channel. The homogeneous metric used was the co-occurrence matrix energy feature COE of the blocks calculated by Equation (1): ) , ( ) , (